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ABSTRACT 


\ 

\ 

This  report  documents  the  1988  application  of  the  Air  Logistics  Early 
Requirements  Technique  (ALERT)  for  estimating  the  FY92-FY97  Program 
Objective  Memorandum  (POM)  inputs  for  the  Peacetime  Operation  Spares 
portion  of  the  aircraft  replenishment  spares  budget  (BP  15).  This  is  the  sixth 
consecutive  year  that  ALERT  has  been  used  to  prepare  AFLC  POM  inputs  to 
the  Air  Staff.  ALERT’s  logic  is  a  combination  of  statistical  forecasts  and 
management  adjustments  to  these  forecasts  which  yield  the  Command’s  total 
peacetime  operating  stock  requirement  for  the  POM  period. 
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EXECUTIVE  SUMMARY 


The  ALERT  model  has  been  used  by  the  Air  Force  Logistics  Command 
since  1984  to  project  the  BP  15  aircraft  peacetime  spares  Program  Objective 
Memorandum  (POM)  requirement.  ALERT  performs  a  statistical  analysis  of 
historical  data  to  forecast  the  BP  15  Peacetime  Operation  Spares  (POS) 
requirement  by  weapon  system.  The  AFLC  BP  15  budget  program  managers 
review  the  statistical  projections  and  make  final  adjustments.  ALERT  is  the 
only  BP15  POM  forecasting  approach  sanctioned  by  Air  Staff.  This  report 
documents  the  development  of  the  FY92-FY97  POM  forecasts.  The  following 
table  shows  the  ALERT  estimates  for  the  total  BP15  budget  for  the  FY92-FY97 
POM.  „  ,  - 

FY92  FY9.^  FY94  FY95  FY96  FY97 

2442.9  2506.7  2566.9  2608.1  2665.1  2719.3 

ALERT  uses  16  individual  weapon  system  regression  equations  to  develop  each 
fiscal  year’s  BP15  estimate  by  weapon  system.  These  individual  forecasts  are 
then  added  together  to  develop  the  Air  Force  total  BP15  POM  forecast. 
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CHAPTER  1 


THE  PROBLEM 

Aircraft  replenishment  spares  (BPI5)  requirement  forecasts  are  needed  by 
weapon  system  for  input  into  the  Program  Objective  Memorandumn  (POM). 
Due  to  a  lapse  of  three  to  seven  years  between  projection  and  initiation  of 
such  requirements  in  the  POM  and  budget  process,  the  Air  Force  needs  a  long 
range  forecasting  technique  to  project  the  outyear  requirements. 

Background 

AFLC  uses  the  ALERT  model  to  forecast  Peacetime  Operating  Stock  (POS) 
BP15  POM  requirements.  The  ALERT  model  is  a  long-range  forecasting  model 
used  to  forecast  BP15  peacetime  operating  stock  requirements  for  the  Air 
Force  POM.  The  once-a-year  data  gathering  process  to  support  ALERT 
depends  heavily  upon  the  timely  receipt  of  the  data  sources,  such  as  the  D041 
Central  Secondary  Item  Stratification  (CSIS)  data,  used  to  develop  the  weapon 
system  forecasts.  Once  developed,  these  forecasts  may  be  adjusted  by  the 
6P1S  budget  manager  based  on  other  information  which  is  not  easily 
incorporated  into  the  statistical  forecast. 

Objectives 

1.  To  run  ALERT  for  forecasting  budget  requirements  for  input  into  the 
FY92-97  POM. 

2.  To  document  the  ALERT  process  for  the  FY92-97  POM. 
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CHAPTER  2 


Wc  document  the  development  of  the  FY92  to  FY97  POM  forecast  in 
three  sections.  First  we  describe  the  ALERT  model.  In  the  second  section  wc 
discuss  the  results  for  the  FY92  to  FY97  forecasts.  In  the  final  section,  we 
discuss  issues. 


ANALYSIS 


The  BP15  budget  program  managers  used  ALERT  to  develop  the  FY92 
through  FY97  POS  POM  requirement.  Wc  document  the  ALERT  model  runs  for 
the  BP  15  POS  POM  in  three  parts.  First,  we  describe  the  model.  Then  we 
present  the  results  for  the  FY92-97  forecasts.  Lastly,  we  discuss  issues. 


Model  Description 


We  describe  the  ALERT  model  in  last  year's  ALERT  forecast  report  [1] 
for  the  FY90-94  POM.  (This  year’s  POM  covers  the  FY92-97  time  period 
instead  of  FY91-95.  This  was  done  to  correspond  to  the  biennial  POM  cycle 
dictated  by  Congress.)  We  repeat  the  description  here  and  update  it  as 
necessary.  We  plan  to  continue  updating  the  ALERT  forecasts  annually. 

ALERT  is  a  multivariate  regression  based  model  that  uses  historical  data 
to  predict  future  POM  requirements  by  weapon  system.  The  model,  which 
previously  resided  on  the  Honeywell  CREATE  computer  system,  now  resides  on 
the  Planning,  Programming,  and  Budgeting  System  (PPBS)  Subproject  5  analysis 
and  forecasting  region  of  the  Requirements  Data  Bank  (RDB)  computer  system 
[3].  The  model  is  programmed  in  the  SAS  statistical  programming  language. 
ALERT  predicts  the  BP  15  Peacetime  Operating  Stock  (POS)  POM  requirement. 
The  model  uses  up  to  eleven  years  of  historical  data  which  includes  the 
following: 

1.  Past  Budget  Estimate  Submissions  for  the  BP15  POS. 

2.  The  sum  of  the  D041  Central  and  Secondary  Item  Stratification  (CSIS) 
extended  year  buy  requirement  plus  the  CSIS  Approved  Force  Acquisition 
Objective  (AFAO)  buy  requirement  for  comdemnations  only  (this  is  the 

estimate  of  recoverable  buy  requirements  three  to  four  years  from  the  current 
June  computation). 

3.  The  reciprocal  of  the  present  and  projected  age  of  the  fleet  (developed 
by  USAF/AC). 

4.  The  dollar  value  of  the  fleet  (developed  by  AFLC/MMM  from  USAF/AC 
data). 

ALERT  is  actually  a  five  step  process. 

Step  1  -  Update  the  ALERT  Data  Base  -  In  this  step,  we  collect  the  most 
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up-tO'datc  data  on  each  of  the  variables.  This  includes  updating  the  D041 
Central  and  Secondary  Item  Stratification  (CSIS)  data  and  the  age  and  value  of 
the  fleet  data  and  entering  them  in  the  data  base. 

Step  2  -  Develop  Regression  Equations  -  In  this  step,  we  run  the  ALERT 
model  on  the  RDB  computer  which  develops  regression  equations,  and  then  we 
select  the  equations  that  result  in  the  best  fit  of  the  data.  The  criteria  used 

to  determine  the  best  fit  consist  of  the  following  statistical  tests;  the  adjusted 
r-square  (the  coefficient  of  determination),  the  F  statistic,  the  residual 
pattern,  and  the  Durbin-Watson  test.  We  documented  these  statistical 
techniques  in  reference  [3]. 

Each  weapon  system  has  its  own  regression  equation  and  its  own  set  of 
independent  variables.  Appendix  A  provides  the  best  set  of  regression 
equations  for  FY91  and  FY92  through  FY97.  We  developed  a  different  set  of 

equations  for  FY92  through  FY97  because  these  forecasts  include  the  FY91 
forecasts  as  historical  data.  (Using  a  regression  forecast  as  another  data  point 
in  developing  yet  another  forecast  is  a  statistical  technique  called 

bootstrapping.) 

Step  3  -  Management  Review  -  The  results  of  step  2  reflect  the  best  set 
of  regression  equations  and  forecasts  based  on  historical  data.  However, 
historical  data  is  not  always  the  best  prediction  of  the  future.  This  is 
because  it  assumes  the  forces  at  work  in  the  past  will  continue  in  a 
predictable  way  into  the  future.  Therefore,  we  include  a  management  scrub 

which  provides  management  a  chance  to  kok  at  the  results  and  identify  any 
known  occurrence  that  would  impact  the  future. 

Step  4  -  Develop  New  Regression  Equations  -  We  repeat  step  2  with  the 
data  updated  from  the  management  scrub,  and  we  include  "dummy"  variables 
to  reflect  future  known  occurrences.  For  example,  if  the  budget  program 
managers  know  of  a  significant  modification  or  program  change,  we  adjust  the 
regression  equation  to  account  for  the  change. 

Step  5  -  Present  the  ALERT  Forecasts  -  At  this  point,  we  present  the 
ALERT  forecasts  by  weapon  system  and  total  BP15  requirements  to  the  budget 
program  manager. 
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Results 


In  Table  1  below,  we  present  the  forecasted  dollar  values  for  the  BP15 
POS  POM  for  FY91.  It  also  shows  the  adjusted  r-square  values.  The  adjusted 
r-squarc  is  known  as  the  coefficient  of  determination  and  is  defined  as  the 
proportion  of  the  total  variation  in  the  dependent  variable  (the  forecasted 
requirement)  that  is  explained  by  the  regression  line.  A  value  of  1.00  means 
that  the  variation  in  the  dependent  variable  is  totally  explained  by  the 
regression  line. 


ALERT  POM  FORECAST 
FY91 


WEAPON 

SYSTEM 

FY91 

FORECAST  (IN  $M) 

ADJUSTED  R-SQUARE 

A-7 

9.3 

0.89 

A-10 

32.8 

0.75 

B-1 

108.1 

0.92 

B-52 

83.1 

0.64 

C-5 

179.1 

0.66 

C-130 

55.1 

0.61 

C-135 

161.4 

0.73 

C-141 

30.7 

0.64 

COMMON 

330.7 

0.77 

£-3 

11.9 

0.83 

F-4 

35.4 

0.73 

F-15 

210.8 

0.17 

F-16 

198.4 

0.85 

F-111 

133.3 

0.80 

FlOO  ENGINE 

569.7 

0.66 

OTHER 

289.0 

0.74 

Table  1 


We  entered  the  FY91  forecasts  in  the  ALERT  data  base  as  historical  data 
and  used  it  to  forecast  the  BP15  POS  for  FY92  through  FY97.  This  is  a 
statistical  technique  known  as  bootstrapping. 

For  the  ALERT  model  runs  for  the  FY92  through  FY97,  we  did  not  use 
the  D041  CSIS  data.  This  is  because  there  is  no  CSIS  data  for  the  POM 
outyears.  In  Table  2,  wc  show  the  FY92  through  FY97  POM  forecasts  by 
weapon  system. 
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ALERT  POM  FORECAST 
FY92  -  FY97 


($  IN  MILLIONS) 

WEAPON  ADJUSTED 


SYSTEM 

FY92 

FY93 

FY94 

FY95 

FY96 

FY97 

R-SQUARE 

A-7 

13.2 

12.8 

12.3 

11.9 

11.4 

11.0 

.75 

A-10 

27.1 

22.4 

17.8 

13.9 

10.2 

6.5 

.74 

B-1 

116.9 

117.8 

118.7 

120.4 

122.1 

123.9 

.89 

B-52 

81.4 

79.8 

78.1 

66.3 

64.7 

63.0 

.65 

C-5 

204.4 

216.9 

229.3 

241.7 

254.2 

266.6 

.70 

C-130 

52.8 

50.7 

55.7 

53.7 

58.7 

63.7 

.62 

C-135 

146.7 

150.8 

155.5 

160.9 

166.5 

172.3 

.76 

C-141 

28.3 

25.6 

23.3 

20.5 

18.3 

16.2 

.63 

COMMON 

333.2 

337.1 

341.0 

344.9 

348.8 

352.7 

.73 

E-3 

18.1 

17.8 

17.5 

17.3 

17.1 

16.9 

.83 

F-4 

11.9 

12.1 

13.9 

16.0 

18.1 

20.3 

.72 

F-15 

196.9 

203.3 

209.2 

225.5 

241.0 

256.5 

.07 

F-16 

218.0 

233.5 

249.1 

264.7 

280.4 

296.0 

.89 

F-111 

135.0 

144.6 

150.9 

137.7 

123.5 

108.2 

.73 

FIDO  ENG 

578.6 

586.4 

593.1 

604.3 

614.3 

622.1 

.68 

OTHER 

280.4 

295.1 

301.8 

308.7 

315.8 

323.4 

.63 

TOTAL 

2442.9 

2506.7 

2566.9 

2608.1 

2665.1  2719.3 

Tabic  2 


When  we  transferred  the  ALERT  model  to  the  RDB  and  reprogrammed  it 
using  SAS,  we  included  several  enhancements  which  we  describe  in  reference 
[3].  In  the  management  review  of  the  ALERT  forecasts,  we  now  provide  the 
BP15  program  managers  with  a  "library"  of  regression  equations  and  their 
associated  forecasts.  The  SAS  statistical  programming  develops  several  such 
equations  for  each  weapon  system  using  different  combinations  of  the  regressor 
variables.  From  these  different  combinations,  SAS  determines  the  central 
tendency  of  the  BP  15  POS  BES  requirement  across  time  and  projects  that  trend 
into  the  future.  The  BP15  program  manager  then  reviews  the  equations,  as 
necessary,  and  selects  those  which  best  reflect  the  future  POS  requirement 
based  on  his  knowledge  of  the  weapon  system,  and  future  occurrences  which 

cannot  be  accounted  for  in  the  historical  ALERT  data  base. 

In  addition  to  .eviewing  the  regression  equations  and  results,  we  review 
the  ALERT  data  i-o  as  necessary,  to  ensure  that  it  is  correct  and  up-to-date 

and  that  all  known  events  are  considered.  For  example,  the  BP  manager  had 
to  subtract  out  norf-*'’-  ring  requirements  for  both  1985.  In  1985,  the  Air 
Force  procurred  Sint^’c  Integrated  Operations  Plan  (SIOP)  additive  spares 
support  kits  as  a  one  time  requirement  for  the  B-1.  Table  2  above  reflects 

the  final  ALERT  regression  results.  In  Appendix  B,  we  have  attached  graphs 

reflecting  the  forecasted  BP15  POM  data  versus  the  actual  BP15  POS  BES 
requirement  (the  dependent  variable  that  ALERT  forecasted  through  the  POM 
outyears). 
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Issues 


We  need  to  ensure  the  integrity  of  the  ALERT  data  base.  Two  inputs  to 
ALERT  had  been  developed  by  USAF/AC.  These  are  the  fleet  value  and  fleet 
age  data.  The  fleet  value  numbers  are  no  longer  being  developed  by  Air  Staff, 
and  will  need  to  be  developed  by  AFLC.  We  need  to  identify  the  AFLC  OPR 
for  this  development  effort  and  ensure  timely  delivery  of  this  input  to  the 

ALERT  model.  Also,  we  had  difficulty  obtaining  the  fleet  age  numbers  from 

Air  Staff  this  year.  We  need  to  ensure  that  Air  Staff  is  aware  of  our 

continuing  need  for  fleet  age  data  as  an  input  to  the  ALERT  modei,  and  that 
it  will  be  provided  to  us  annually  and  in  a  timely  fashion. 

In  the  future,  the  Air  Force  will  need  more  accurate  POM  forecasts  at 
the  weapon  system  level.  Today,  the  funding  within  the  budget  program  is  not 

specifically  tied  to  one  weapon  system.  Portions  of  the  total  BP  15  requirement 

can  be  shifted  from  one  weapon  system  to  another  as  conditions  may  dictate. 
However,  due  to  an  Air  Force  decision,  this  will  no  longer  be  the  case.  In 
the  future,  funding  within  a  specific  budget  program  (such  as  BP15)  will  be 

tied  by  the  budget  managers  to  the  individual  weapon  system.  In  a  previous 
report  on  ALERT,  we  found  that  individual  weapon  system  forecasts  can  be 
very  inaccurate  when  compared  to  the  total  rolled  up  BP  15  forecast  [2].  That 
is,  although  individual  weapon  system  requirements  could  be  over  or  under 
forecasted,  these  fluctuations  tended  to  cancel  themselves  out  when  rolled  up 
into  an  overall  budget  program  forecast.  We  need  to  analyze  the  ALERT 
forecasts  for  individual  weapon  systems  and  develop  ways  to  improve  the 

forecasts. 

One  final  issue  identified  by  the  BP15  program  manager  is  to  develop  an 
ALERT  forecast  for  the  total  POS  BP15  program.  Currently,  ALERT  develops 
POM  forecasts  by  weapon  system  and  rolls  them  up  for  the  total  BP15 
requirement.  The  BP  15  program  manager  wants  a  forecast  developed  for  the 
entire  BP  15  program.  This  would  serve  as  a  validity  check  for  the  current 
ALERT  BP15  POM  forecasts.  We  need  to  develop  a  total  POS  ALERT  BP15 
POM  forecast  based  on  a  single  statistical  function  as  opposed  to  the  current 

aggregated  ALERT  POM  forecast. 
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CHAPTER  3 


CONCLUSIONS  AND  ACTIONS 


Conclusions 

1.  AFLC  uses  the  ALERT  model  to  forecast  Peacetime  Operating  Stocks  (POS) 
BP  15  POM  requirements. 

2.  The  BP15  budget  program  manager  used  ALERT  to  develop  the  FY92 

through  FY97  POS  POM  requirement. 

3.  The  fleet  value  data  required  by  ALERT  are  no  longer  developed  by  Air 

Staff. 

4.  We  need  to  have  fleet  age  data  provided  to  AFLC/MMM  in  a  timely 

manner  for  input  to  the  ALERT  model. 

5.  The  Air  Force  will  need  more  accurate  POM  forecasts  at  the  weapon 

system  level. 

6.  ALERT  develops  POM  forecasts  by  weapon  system  which  arc  then  rolled  up 
into  a  total  BP15  requirement. 

7.  We  need  a  separate  ALERT  forecast  of  the  total  BP15  requirement  to  use 

as  a  cross-check. 

Ag.tigns 

1.  Continue  to  document  the  annual  ALERT  forecasts.  (OPR:  HQ 

AFLC/MMMA  and  MMMI) 

2.  Develop  fleet  value  data  for  input  to  the  ALERT  model.  (OPR:  HQ 

AFLC/ACC,  OCR:  HQ  AFLC/MMM) 

3.  Ensure  that  fleet  age  data  is  received  annually  in  a  timely  fashion  for 
input  to  the  ALERT  model.  (OPR:  HQ  AFLC/ACC,  OCR:  HQ  AFLC/MMM) 

4.  Analyze  the  ALERT  forecasts  for  individual  weapon  systems  and  develop 

ways  to  improve  the  forecasts  as  necessary.  (OPR:  HQ  AFLC/MMMA,  OCR: 
MMMI) 

5.  Develop  a  total  POS  BP  15  ALERT  POM  forecast.  (OPR:  HQ  AFl.C/MMMA, 
OCR:  MMMI) 
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Appendix  A 

ALERT  Regression  Equations 

In  this  appendix  we  present  the  regression  equations  used  to  forecast  FY91 
and  FY92  through  FY97.  In  Table  A-1  below  are  the  equations  for  the  FY91 


forecast, 

ALERT  POM  FORECAST  -  FY91 
REGRESSION  EQUATIONS 

WEAPON  ADJUSTED 

SYSTEM  REGRESSION  EQUATIONS  R-SQUARE 

A-7  POSBES  =  498.6637  -  1.8273E-05  (TBUY)  -  7.3361E-03  (VALUE)  -  4.6281  (YEAR)  0.8883 

-  691.5611  (AGEREC) 

A-10  POSBES  =  733.0991  -  7.5154  (YEAR)  -  1.0790E-05  (TBUY)  -  149.0764  (AGEGRE)  0.7458 

B-1  POSBES  =  289.00256  -  1.9883  (YEAR)  0.9219 

B-52  POSBES  =  1484.1365  -  11.7438  (YEAR)  -  10070.4673  (AGEREC)  0.6441 

C-5  POSBES  =  5.6870  +  6.2327E-03  (VALUE)  0.6609 

C-130  POSBES  =  1214.8462  -  6.1750E-08  (TBUY)  -  7818.1042  (AGEREC)  -  8.4362  (YEAR)  0.6057 

C-136  POSBES  =  -1069.1878  -  4.6311E-05  (TBUY)  +  13.7630  (YEAR)  0.7256 

C-141  POSBES  =  996.4131  -  3.7182E-05  (TBUY)  -  9.0238  (YEAR)  -  3356.3727  (AGEREC)  0.6448 

COMMON  POSBES  =  246.9300  +  4.5696E-05  (TBUY)  0.7666 

E-3  POSBES  =  245.8246  -  1.9112E-0S  (TBUY)  -  2.5442  (YEAR)  0.8323 

F-4  POSBES  =  -549.91562  -  12.9867E-06  (TBUY)  +  1.3993E-02  (VALUE)  +  6.4224  (YEAR)  0.7337 

F-16  POSBES  =  -740.6069  -  4.2228E-06  (TBUY)  +  10.7968  (YEAR)  0.1696 

F-16  POSBES  =  -1091.8790  +  6.8753E-06  (TBUY)  +  14.1240  (YEAR)  0.8512 

F-111  POSBES  =  3786.83042  +  7.3826E-02  (VALUE)  -  57.3648  (YEAR)  0.7962 

FlOO  POSBES  =  713.6608  -  1114.9940  (AGEREC)  0.7352 

ENGINE 

OTHER  POSBES  =  -4495.3487  -  2.6264E-08  (VALUE)  +  60.9694  (YEAR)  0.7352 

Table  A-1 


The  adjusted  r-square  value  in  Tables  A-1  and  A-2  are  the  coefficient  of 
determination.  It  is  defined  as  being  the  proportion  of  the  total  variation 
in  the  dependent  variable  (POSBES)  that  is  explained  by  the  regression  line. 


10 


In 

forecast 


WEAPON 

SYSTEM 

A-7 

A-10 

B-1 

B-S2 

C-5 

C-130 

C>1S5 

C-141 

COMMON 

E-3 

F-4 

F-15 

F-16 

F-m 

FlOO 

ENGINE 

OTHER 


Table  A-2  below  are  the  regression  equations  for  the  BP  15 
of  FY92  through  FY97. 

ALERT  POM  FORECAST  -  FY92  -  FY97 
REGRESSION  EQUATIONS 


REGRESSION  EQUATIONS 

POSBES  =  S4.21346448  -  0.44583652  (YEAR) 

POSBES  =  513.64993  +  .01627106  (VALUE)  -  6.431657668 
POSBES  =  43.33600081  +  0.002927883  (VALUE) 

POSBES  =  1484.49635  -  11.74718849  (YEAR)  -  10072.25904  (AGEREC) 
POSBES  =  -938.91306  +  12.42786121  (YEAR) 

POSBES  =  1222.95187  -  9.17000568  (YEAR)  -  7098.04283  (AGEREC) 
POSBES  =  -72.05442266  +  .008191279  (VALUE) 

POSBES  =  559.44986  +  0.02118041  (VALUE)  -  8.32407571  (YEAR) 
POSBES  =  -26.41598214  +  3.89794643  (YEAR) 

POSBES  =  14.17483254  +  46.17381933  (AGEREC) 

POSBES  =  -128.31952  +  .006294832  (VALUE)  +  1.05951761  (YEAR) 
POSBES  =  -1662.54344  -  0.006422922  (VALUE)  +  23.06669361  (YEAR) 
POSBES  =  -1232.61808  =  15.74490816  (YEAR)  +  12.16614263  (AGEREC) 
POSBES  =  2146.99042  -  0.06473493  (VALUE)  -  10747.71479  (AGEREC) 
POSBES  =  713.54662  -  1114.98240  (AGEREC) 

POSBES  =  13.59294072  +  0.000837384  (VALUE) 

Table  A-2 


POM  POS 

ADJUSTED 

R-SQUARE 

0.7549 

0.7446 

0.8938 

0.6540 

0.6992 

0.6161 

0.7637 

0.6286 

0.7347 

0.8296 

0.7238 

0.0736 

0.8865 

0.7286 

0.6829 

0.6278 
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Appendix  B 


Graphical  Results  of  ALERT  FY92-97 
BP15  POM  POS  Requirement 


In  this  appendix,  we  present  the  graphical  results  of  the  ALERT  POM 
forecast  for  BP  15  peacetime  operating  stock  requirements.  These  graphs 

reflect  the  forecasted  BP  15  POM  (ALERT)  data  versus  the  actual  BP  15  POS  BES 
requirement  by  weapon  system  (obtained  from  the  annual  BP  15  BES  reports 
submitted  to  Air  Staff).  Also  included  in  these  graphs  are  the  upper  and 

lower  95  percent  confidence  limits  of  the  forecasted  numbers.  The 

confidence  limits  can  be  interpreted  as  follows:  for  such  forecasts  as  are 

presented  in  the  graphs,  95  percent  of  the  time  we  expect  the  predicted 

value,  when  it  actually  occurs,  to  fall  within  the  upper  and  lower  limit. 
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